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Abstract 

In the study of large scale stochastic networks with resource management, differ- 
ential equations and mean-field limits are two key techniques. Recent research shows 
that the expected fraction vector (that is, the tailed probability vector) plays a key 
role in setting up mean-field differential equations. To further apply the technique of 
tailed probability vector to deal with resource management of large scale stochastic 
networks, this paper discusses tailed probabilities in some basic queueing processes 
including QBD processes, Markov chains of GI/M/1 type and of M/G/I type, and 
also provides some effective and efficient algorithms for computing the tailed proba- 
bilities by means of the matrix-geometric solution, the matrix-iterative solution, the 
matrix-product solution and the two types of i?G-factorizations. Furthermore, we con- 
sider four queueing examples: The M/M/l retrial queue, the M(n)/M(n)/I queue, the 
M/M/l queue with server multiple vacations and the M/M/l queue with repairable 
server, where the M/M/l retrial queue is given a detailed discussion, while the other 
three examples are analyzed simply. Note that the results given in this paper will be 
very useful in the study of large scale stochastic networks with resource management, 
including the supermarket models and the work stealing models. 

Keywords: Randomized load balancing; supermarket model; work stealing model; 
QBD Process; Markov chain of GI/M/1 type; Markov chain of M/G/l type. 
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1 Introduction 



We consider a discrete-time (resp. continuous-time) Markov chain whose transition prob- 
ability matrix (resp. infinitesimal generator) is given by 
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where the size of the matrix Po,o is m 0: the size of the matrix Pjj is m for j > 1, and 
the sizes of other matrices can be determined accordingly. We assume that the Markov 
chain P is irreducible, aperiod and positive recurrent. Let x = (xq,x\,X2,xs, . . .) be the 
stationary probability vector of the Markov chain P, where the size of the vector xq is 
m,Q while the size of the vector Xj is m for j > 1. The main purpose of this paper is to 
discuss the tailed probabilities: ir^ = Sj=fc x j an d to provide some effective and efficient 
algorithms for computing the tailed probabilities ir^ for k > 1. 

Recent queueing literature indicates that the study of tailed probabilities {iTk,k > 1} 
plays a key role in analyzing large scale stochastic networks with resource management, 
such as, the supermarket models and the work stealing models, e.g., see Vvedenskaya and 
Suhov [39], Mitzenmacher |27| . Ethier and Kurtz |10| and Kurtz |14| . When considering 
a large scale stochastic network with resource management, differential equations and 
mean-field limits are two key techniques, while the tailed probabilities is a key in setting up 
mean-field differential equations. The detailed interpretation on the mean-field differential 
equations is given by Vvedenskaya, Dobrushin and Karpelevich |38j and Mitzenmacher 
|25j. In those two papers, the authors considered a supermarket model with N identical 
servers, where the service times are exponential with service rate fj,, and the input flow 
is Poisson with arrival rate NX. Upon arrival, each customer chooses d > 1 servers from 
the N servers independently and uniformly at random, and joins the one whose queue 
length is the shortest. Let ni ,{t) denote the number of servers queued by at least k > 



customers at time t, and (t) = lini/v- 
supermarket model is stable and 



E 



n 



(TV) 



(t)/N . If p = X/n < 1. Then the 



d_ 



Uk (t) = xUuk-i {t)] d - [u k (t)] d j - M [uk (t) - u k+ i (t)] 



(1) 



with the boundary condition uq (t) = 1. We write that tt^ = lrnn,_j, +00 Uk (t) for k > 1. 
Then ttq = 1 and for A; > 1 

A(irjU-7rg)-M(7r fc -7r fc+1 ) = 0. (2) 

This gives 

d k -l 

i^k = 9 d - x , k>l. 

If d = 1, then 7Tfc = /0 fc for A; > are the tailed probabilities of the M/M/T queue. 

Since the introduction of the expected fraction vector (or the tailed probability vector) 
by Vvedenskaya, Dobrushin and Karpelevich |38j or Mitzenmacher |25| . research on su- 
permarket models and work stealing models has been greatly motivated by some practical 
applications such as computer networks, manufacturing systems and transportation net- 
works. Subsequent papers have been published on this theme, among which, see, modeling 
more crucial factors by Mitzenmacher [261 127] . Jacquet and Vvedenskaya [12], Jacquet, 
Suhov and Vvedenskaya [13j and Vvedenskaya and Suhov [40]; studying fast Jackson net- 
works by Martin and Suhov [24J, Martin [23J and Suhov and Vvedenskaya [36]; discussing 
value of information by Mitzenmacher [28] and Mitzenmacher, Prabhakar and Shah [29J; 
analyzing non-exponential server times and/or non-Poisson inputs by Mitzenmacher [25], 
Vvedenskaya and Suhov [39J, Bramson, Lu and Prabhakar [U [5j [6] , Li, Lui and Wang 
[20J , Li and Lui [19] and Li [T7] . For a comprehensive analysis of supermarket models and 
work stealing models, readers may refer to Vvedenskaya and Suhov [39], Mitzenmacher, 
Richa and Sitaraman [30 j and Mitzenmacher and Upfal [31J. 

During the last two decades considerable attention has been paid to studying QBD 
processes, which has been well documented, for example, by Chapter 3 of Neuts [33j , 
Naoumov [32] , Bright and Taylor [7J [8] , Ramaswami |35] , Latouche and Ramaswami |15] 
and Li and Cao [18j. For Markov chains of GI/M/1 type and Markov chains of M/G/l 
type, readers may refer to four excellent books by Neuts [331 [M], Latouche and Ramaswami 
[E] and Li [16]. 

Some papers were published on asymptotic behavior of the stationary probability vec- 
tors for both queueing systems and Markov chains. Readers may refer to, such as, Markov 
chains of GI/M/1 type by Neuts [33]; Markov chains of M/G/l type by Falkenberg [TT] . 
Abate, Choudhury and Whitt [1], Choudhury and Whitt [9j, Asmussen and M0ller [2] and 
Takine [37] ; and Markov chains of GI/G/1 type by Li and Zhao |2H 122] . 
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The main purpose of this paper is to provide some novel, effective and efficient algo- 
rithms for computing the tailed probabilities in three classes of important Markov chains: 
QBD processes, Markov chains of GI/M/1 type and Markov chains of M/G/l type. Note 
that the algorithms are based on the matrix-geometric solution, the matrix-iterative so- 
lution, the matrix-product solution and the two types of i?G-factorizations. Also, we 
consider four queueing examples: The M/M/l retrial queue, the M(n)/M(n)/1 queue, the 
M/M/l queue with server multiple vacations and the M/M/l queue with repairable server. 
Based on this, it is seen that the method of this paper can deal with more general queue 
examples, for example, the MAP/PH/1 queue, the GI/PH/1 queue and the BMAP/SM/1 
queue. Therefore, the results of this paper are very useful in the study of large scale 
stochastic networks with resource management, such as, the supermarket models and the 
work stealing models. 

The remainder of this paper is organized as follows. In Section 2, we analyze a 
continuous-time level-independent QBD process. When the QBD process is irreducible, 
aperiodic and positive recurrent, we apply the matrix-geometric solution and the two 
types of i?G-factorizations to compute the tailed probabilities of its stationary probabil- 
ity vector. In Section 3, we consider an continuous-time level-dependent QBD process, 
and compute the tailed probabilities of its stationary probability vector. In Section 4, we 
discuss two classes of important Markov chains: Markov chains of GI/M/1 type and of 
M/G/l type, and derive the tailed probabilities of their stationary probability vectors. In 
Section 5, we study four queueing examples, where the M/M/l retrial queue is given a 
detailed discussion, while the other three queues are analyzed simply. Some concluding 
remarks are given in Section 6. 

2 Level-Independent QBD Processes 

In this section, we consider a continuous-time level-independent QBD process. When the 
QBD process is irreducible, aperiodic and positive recurrent, we apply the two types of 
i?G-factorizations to compute the tailed probabilities of its stationary probability vector. 
Furthermore, the tailed probabilities of the stationary probability vector is well related to 
the matrix-geometric solution when the two types of i?G-factorizations are used. 

We consider a continuous-time level-independent QBD process whose infinitesimal gen- 
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erator is given by 
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( B 1 B 
B 2 A x A 

A 2 A x A Q 

A 2 Ax A 



(3) 



/ 



where the sizes of the two matrices Bx and Ax are mo and m, respectively; and the sizes 
of other matrices can be determined accordingly. We assume that this QBD process is 
irreducible, aperiodic and positive recurrent. 



2.1 The matrix-geometric solution 

Let x = (xo,xx,x 2 , • • •) be the stationary probability vector of the QBD process, and R 
and G the minimal nonnegative solutions to the nonlinear equations Aq + RAx +R 2 A 2 = 
and A G 2 + A\G + A 2 = 0, respectively. Then 

x k = xxR k -\ k>2, (4) 

where xq and xx are uniquely determined by the following system of linear equations 

xqBi + xxB 2 = 0, 
x B + xx (Ax + RA 2 ) = 0, 
x$e + x\ (I — R)^ 1 e = 1. 



We write 



It follows from (HI) that 



Kk = ^2xj, k>i. 

j=k 

TT k =x 1 (I-R)~ 1 R k ~ 1 , k>l. 



(5) 



(6) 



2.2 The UL-type i?G-factorization 

Now, we apply the UL-type i?G-factorizations to provide a novel method for deriving the 
tailed probability vector tt = (iri, 17%, 173, . . .). 
Note that xQ = 0, we have 



x B + xxAx + x 2 A 2 = 0, k = l, 

Xk-iAo + x k Ax + x k+l A 2 = 0, k > 2, 



(7) 



This gives 



Hence we obtain 



where 



iri (A + A x ) + -n 2 A 2 = -x B , k = 1, 
x fc _iA + + x k+1 A 2 = 0, fc > 2. 



vrQ = -(x S , 0,0,0 
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Let 



<S> = A + Ax + RA 2 , 
<S> k = $ = Ax + RA 2 , k>l. 
Then the UL-type i?G-factorization of the matrix Q is given by 

Q = (I-Ru)U(I-G L ), 



where 
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It follows from (JHJ) and © that 

7T = - (x S , 0, 0, 0, . . .) (7 - Gl)- 1 C/" 1 (/ - i^)" 
= -(xoBo^o 1 ' 5 5 ° ; ---)( / - i? c/)" 1 - 



Note that 
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we obtain 



7Ti = x B (-^q 1 ) , k = l, 

n k = x Bo R k -\ k>2. 

Comparing (|10p with ([6]), we obtain 

x B (-$ 1 ) =xi (I-R)' 1 . 



(10) 



(11) 



2.3 The LU-type i?G-factorization 

In what follows we apply the LU-type i?G-factorizations to provide a novel and effective 
method for deriving the tailed probability vector it = (tti, 7:2, 7T3, . . .). 
Let 

fo = A) + Ai 

and for k > 1 

y k = A 1 +A 2 (-^-\)A . 

We write that for k > 1 

R fc = A 2 (-t^) 

and 

Gw = (-»£ 1 )4 
Then the LU-type i?G-factorization of the matrix Q is given by 



where 



Ri 



: (J - R L ) U (/ - Gtf) , 
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R 3 
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U = diag(*o,*i,*2,...) 
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= GiGi +1 G l+2 ■ ■ ■ G, +fc _i, k>l,l>0. 
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It follows from © and Q that 



7T = - (x 0J B , 0, 0, 0, ...)(/ - Gc/)- 1 IT 1 (/ - i^r 1 



this gives 



and n > 2 



-In 



fc=i 



(n-1)- 



3 Level-Dependent QBD Processes 



In this section, we consider a continuous-time level-dependent QBD process. When the 
QBD process is irreducible, aperiodic and positive recurrent, we apply the LU-type RG- 
factorization to compute the tailed probabilities of its stationary probability vector. Sim- 
ilarly, we can apply the UL-type i?G-factorization to compute the tailed probabilities 
without any difficulty. 

We consider a continuous-time level-dependent QBD process whose infinitesimal gen- 
erator is given by 
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AF A 
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(2) 4 (2) 
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(15) 



where the size of the matrix A^ is uiq while the size of the matrix A^ is m for k > 1. 
We assume that this QBD process is irreducible, aperiodic and positive recurrent. 



3.1 The matrix-product solution 

Let the matrix sequence {Ri, I > 0} be the minimal nonnegative solution to the system of 
nonlinear matrix equations 

4° + RiAf +l) + RiR l+1 A { ^ 2) = 0, I > 0. 



Using Chapter 1 of Li [16] , we have 

Xo = kv, (16) 

Xfc = kvRqRi ■ ■ ■ Rk-i, k>l, (17) 

where K is a normalization constant, v is the stationary probability vector of the censored 
chain U = A^ + RqA^ to level 0. Therefore, it follows from (JTSJ) and (dZJ) that 

oo 

vTfc = xj = kvRqRi ■ ■ ■ Rk-i (I + Rk + RkRk+i + RkRk+iRk+2 H ) • 

j=k 

3.2 The LU-type RG- factorization 

Here, we only provide a detailed analysis for applying the LU-type i?G-factorization to 
compute the tailed probabilities, while the UL-type i?G-factorization can be used similarly 
for such an analysis. 
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Since 

CO 

TTfc = y^.Xj, fc > 1, 

j=k 

it is easy to see that x k = -Kj, — Hk+1 for k > 1. Note that xQ = 0, we have 



A 



(2) 



A 



(k) 



(i) 



-xo4 0) , 



A, 



(fc-1) 



A. 



(fc+1) 



vr fc+2 4 fc+1) 



fc = 1, 
0, fc > 2, 
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This gives 



where 



Thus we obtain 



7r(Q-Q) = -(xo4 0) , 0,0,0,...), 



(18) 




7T (J - QQ-y = - (xo4 0) , 0, 0, 0, . . .) Q~l x , 

where Q^ x is the maximal non-positive inverse of the infinitesimal generator Q. Hence, 
this gives 

CO 

tt = - (so A{, 0) ,0,0,0,...) Q m L £ (QQ-y k ■ 



k=0 



Now, we apply the LU-type i?G-factorization to provide the maximal non-positive 
inverse of the matrix Q. To that end, we write 

*o = A[°) 



and for k > 1 



(fc-i) 
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It is easy to check that ^/ is the infinitesimal generator of an irreducible continuous-time 
Markov chain, and the Markov chain ^/ is transient. Thus the matrix *f>i is invertible for 
I > 0. 

Based on the [/-measure {^i}, for k > 1 we can respectively define the LU-type R- 
and G-measures as 

R fc = 4 fe) (-*-^) 

and 

G fe _ 1 = (-^ 1 )4 fe - 1 \ 

Note that the matrix sequence {R& : k > 1} is the unique nonnegative solution to the 
system of nonlinear matrix equations 

with the boundary condition 

Ri = 4 X) (-*o x ) • 

Hence we obtain 

R fe+ i = -4 fc+1) [n k At 1] + 4* 

Similarly, the matrix sequence {Gfc : A; > 0} is the unique nonnegative solution to the 
system of nonlinear matrix equations 

4 fc) + 4 fe) G fc + 4 fe) G fe _ 1 G fc = 0, 

with the boundary condition 

Go = (-*o 1 )4° ) - 

Thus we obtain 



Gt 



4 fc) +4 fc) G fe _! 



A 



(k) 



The LU-type i?G-factorization of the QBD process is given by 

Q = (I- R L ) U d (/ - G v ) , 

where 



(19) 
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Hence we obtain 

oo 

tt = - (xo4 0) . 0, 0, 0, . . .) Q m L ^ (QQ-L)" 

fc=0 

oo 

= - (xo4 0) > °. °> °> ■ ■ •) ( J - m -1 u d x ( J - ^r 1 E [q ( j 



k=0 



This can be calculated by some ordinary matrix computation. 
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4 Two Classes of Important Markov Chains 

In this section, we consider two classes of important Markov chains: Markov chains of 
GI/M/1 type and of M/G/l type, each of which is basic in the study of queueing processes, 
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e.g., see Neuts [33J GS] for more details. We provide two different methods to derive the 
tailed probabilities of stationary probability vectors of the two classes of Markov chains. 



4.1 Markov chains of GI/M/1 type 

We consider a discrete-time Markov chain P of GI/M/1 type whose transition matrix is 
given by 
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where the sizes of the two matrices B\ and A\ are uiq and m, respectively, while the sizes 
of other matrices can be determined accordingly. We assume that this Markov chain is 
irreducible, aperiodic and positive recurrent. Let the matrix R be the minimal nonnegative 
solution to the nonlinear matrix equation R = ^2^L R k A^. 

In what follows we provide two methods to derive the tailed probabilities of the sta- 
tionary probability vector. 

(a) The matrix-geometric solution 

Using Chapter 2 of Li [16], the stationary probability vector x = (xq, x±, x 2 , ■ ■ ■) is 
given by 

£o = Ty , 

Xk = xqRiR^ 1 , k>l, 



where 



and 



-i 



Rl= \I-J2 RkB k+l) B 



k=0 



tf = X> fc #fc+i, 



k=0 



Ho is the stationary probability vector of the censored Markov chain to level and the 
scalar r is determined by Ylfc=o x ^ e = 1 uniquely. Thus, for k > 1 we have 



n k = Y^ xj = x oRi {i - By 1 B^ 1 

j=k 



(23) 



13 



(b) The UL-type RG-factorization 

Note that x = xP, we obtain 



this gives 



where 



7Ti = 7ri (A + Ai) + n k A k + x B , k = 1, 

k=2 

CO 

^k= Yl Kk-i+jAj, k>2, 
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ir = irF + (x B , 0,0,0,...), 



(24) 
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which is of GI/M/1 type. Then using Chapter 2 of Li [16], the [/-measure is given by 

oo 

* = A + Ax + J2 R i Rk ~ 2A k 

and for k > 1 



k=2 



k=l 



the i?-measure is given by 



R k = R, k > 1, 



and the G-measure 



G jt0 = (i-*r 1 [ J2 Rh ' lA k\ > i>i> 



and 



G^CJ-*)- 1 ^ R^Ak) , j>l. 
\fc=j+i 

Thus, the UL-type i?G-factorization is given by 



I-F=(I-Ru)(I- $ D ) (/ - G L ) , 



(25) 
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It follows from (El and (E5D that 



r 1 

' mm 



tt = (aj B 0j 0,0,0,...) (J 
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we obtain 



ir k = x Bo(I-^ r 1 R k -\ k>l. 
Comparing ([26]) with (J23j) , we obtain 

x B (I -^o)- 1 = x R! (I-Ry 1 - 
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4.2 Markov chains of M/G/l type 

We consider a discrete-time Markov chain P of M/G/l type whose transition matrix is 
given by 
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where the sizes of the two matrices B\ and A\ are mo and m, respectively, while the sizes 
of other matrices can be determined accordingly. We assume that this Markov chain is 
irreducible, aperiodic and positive recurrent. Let the matrix G be the minimal nonnegative 
solution to the nonlinear matrix equation G = Y^k=o A k G k . 

In what follows we provide two methods to derive the tailed probabilities of the sta- 
tionary probability vector. 

(a) The matrix- iterative solution 

Using Chapter 2 of Li [16] , the U -measure is given by 



B 1 + J2B k G k ~ 2 G 1 



k=2 



and for k > 1 



and the i?-measure 



* = Vk = Y,A k G k - 1 ; 



k=l 



and 



0,j 



B k G k - l \{I-^)-\ j>l, 

,fc=i+i 



R i 



A k G k - x (J-*)" 1 , i>l. 
K k=j+i ) 

The stationary probability vector x = (xq,x\,x 2 , ■ ■ ■) is given by 



xo = Ty , 



k-l 



X k = X Ro,k + Yl x i R k-i, k>l, 
i=l 
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where yo is the stationary probability vector of the censored Markov chain \&o to level 
and the scalar r is determined by ^/S=o x k e = 1 uniquely. Thus, we obtain 



oo oo 



j=k j=k i=l j=k 



(b) The UL-type RG-factorization 

Note that x = xP, for k > 1 we obtain 



fe+i 



this gives 



where 



TTfc = XO ^2 B 3 +S ^Z n i A k+l-i + TTl Aj, 
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Using Chapter 2 of Li |16| , the U -measure is given by 

oo oo oo 

3=1 k=2 j=k 



and for k > 1 



and the i?-measure 



fc=i 



(28) 
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B k G k - l \ (J-*)" 1 , j>l, 

.fc=j+l 



and 



^ AfcG*- 1 (J-*)" 1 , j>l. 
v fc=i+i / 

Thus, the UL-type i?G-factorization is given by 



/ - P = (I - R v ) (I - * D ) (/ - G L ) 



(30) 
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where 
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It follows from ([29j) and ([30]) that 



7T 



J=2 i=3 j=4 

oo oo oo 

j=2 j=3 j=4 



5 Some Queueing Examples 

In this section, we consider four queueing examples which indicate how to use our above 
results. We first provide a detailed discussion for the M/M/l retrial queue with exponential 
retrial times. Then we simply analyze other three queueing examples: The M(n)/M(n)/1 
queue, the M/M/l queue with server multiple vacations and the M/M/l queue with 
repairable server. 



5.1 The M/M/l retrial queue 

We consider an M/M/l retrial queue with exponential retrial times, where the arrival, 
service and retrial rates are A, [i and 9, respectively. We denote by N (t) and C (t) the 
number of customers in the orbit and the state of server at time t, respectively, where 
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iV (t) = 0, 1, 2, . . . and C (t) = W for the busy server or / for the idle server. For k > 0, 
we write 

Pw,k (t) = P{C(t) = W,N(t) = k} 

and 

Pl,k (t) = P{C(t) = I,N(t) = k}. 

Hence, we obtain 

-TiPWfl (t) = - (A + /i) p Wfi (t) + Xp Ifi (t) + Qp IX (t) , 
at 

foPw,k (t) = - (A + /i) p w ,k (t) + Xpi,k (t) + (k + l) 0pi,k+i (t) + \pw,k-i (t) , k> 1, 

(31) 



faPifl (*) = Wwfi (t) - \pi,o (t) , 

faPi,k (*) = VPw,k (t) - \pi,k (t) ~ k6p Iik (t), k > 1. 



(32) 



Let 



Qw,k (t) = ^2 pw,i (*) » (*) = E Pl ' 1 (*) • 



Then it follows from (1311) that for A; > 1 



z=fc 



^Qw.fe (t) = A [Qw,fc-i (*) - Qw,k (*)] - /^Qvi/,fc (t) + AQ/,fe (i) + ^ j'p/j (i) 



Since 

oo oo oo 

E ^ (*) = E j [Qjj (*) - Q^+i (*)] = ( fc + !) Q/,fc+i (*) + E ^ (*) 

j=fc+l j=fc+l j=k+2 

we obtain that for k > 1 



-r:Qw,k (t) —A [Qw/fc-i (t) - Qw.fc (*)] _ nQw,k (t) + XQi k (t) 
at 



+ 9 



+ 1) Qj.fc+1 (*) + E Gij(f) 



(33) 



Similarly, it follows from (j32| that for > 1 
d 



dt 



Qi,k (t) = nQw,k (t) - XQ Ik (t) - 



kQi, k (t)+ E QuW 
j=k+l 



(34) 
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Specifically, we have 



Qi,o (*) = vQw,o (*) - AQ/,0 (t) - 9 



i=i 



with the boundary condition 

Qm/,o (*) + Q/,o (f) = l. 

For the M/M/l retrial queue, for k > we write the tailed probabilities 



Note that for I > 



7Tw ife = lim Qw fc (t) , ir I k = lim Q 7 k (t) 

t— >+oo t— >+oo 



—Q Wtl (t) = 0, Aq 7) , (t) = 0, 
it follows from ([33]) to ([36]) that a system of linear equations is given by 

oo 

/■wrw.o - A7T/ 5 o - 6* E 717 J = °' 

KW,0 + 7T/,0 = 1, 



and for k > 1 

A (7TW,k-l - 7TW,fc) - A"TW,fc + A7T/ 5 fc + # 

and 



(k + 1) 7T/ ifc+ i + ^ 7T/^ 



^W,k - ^I,k - 9 



j=k+i 



j=k+2 



0. 



Let 



where 



n = (ni,n 2 ,iij 



)***/) 



Q 





= (^W,k,^I,k) 


k > 1, 




/A! 


C 






\ 




A 2 


C 






£> 


#3 


A 3 C 








D 


B 4 A A 


C 




D 


D 


D B 5 


A 5 C" 












• / 
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where for k > 1 



u ! - (A + " ) " ).B M= 

A -(X + k9) \ (k + l)9 -6 





fx 


0^ 




^0 \ 


H 












V 






V -J 



Note that 7rw,o = P and 717,0 = 1 — p, it follows from ([39]) and (}4"0j) that 

nQ = (-Ap,0,0,...). 
To compute the tailed probabilities, we write 



El 





(k - 1)6 



k > 2, 



( A x C 

E 2 A 2 C 

E 3 A 3 C 

E A A 4 C 
E 5 A b 

V 



\ 



c 



and 



A 





-D 

-D -D 

-£> -D -D 

-D -D —D -D 



V : 



/ 



(42) 



It is easy to see that the matrix Q is the infinite-small generator of an irreducible QBD 
process. Hence, using (1.21) in Chapter 1 of Li [16] we have the UL-type i?G-factorization 
as follows 

Q = (I- R v ) U D (I - G L ) , 
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where 



Ru 



R 
Rx 

R 2 

R 5 



\ 



■ J 



and 



U D = dmg(U ,U 1 ,U 2 ,U 3 ,...) 





Gi 

G 2 

G 3 



V 



/ 



Since Qe = (0, — 9; 0, —0; 0, —6*; 0, —8; . . .) ^ 0, the matrix Uj is invertible for j > 0. This 
gives 

Q^ x = {i-G L y 1 u D l {i-R u r\ 

where 

l^ 1 = diag (U \ U{\U 2 \ C/3- 1 , . . .) . 
It follows from ([H]) and (@2J) that Q = Q - A and 



n(Q-A) = (-A/5,0,0 



)•••/; 



thus we obtain 



Let 



n(/-AQ m L) = (-Ap,0,0,...)Q 

OO 

(/-AQ m 1 ax )" 1 = ^(AQ- 1 ax ) n . 



-1 

max - 



(43) 



n=0 



Then using (|43|) we get 

OO 

n = (-A P ,0,0, . . .) Q'LE ( AQ m 1 ax) n 

n=0 

00 

= (-Xp, 0, 0, ...)(/ - G L )- X [/ d 1 (/ - Ru)" 1 J] [A (J - GzT 1 C^ 1 (/ - it^)" 1 



n=0 



This can be calculated by some ordinary matrix computation. 
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5.2 The M(n)/M(n)/1 queue 

We consider an M(n)/M(n)/1 queue whose arrival and service rates depend on the number 
of customers in this system, denoted as A n and fi n , respectively. We denote by N (t) the 
number of customers in this system at time t. Then N (t) = 0,1,2, — For k > 0, we 
write 

Q k (t) = P {N (t) > k} 
and when the M(n)/M(n)/1 queue is stable, 

7r fc = lim Q k (t) . 

Then we obtain that for k > 1 

with the boundary condition ttq = 1. Let ii\ 
Then 

TTl - 7T 2 = p (7T " 

and for k > 2 

TTfc - TTfc+1 = Pfe-1 (TTfc-1 - TTfc) = Pk-lPk-2 " " " Pl/>0 (1 ~ 5") ■ 

We obtain 

5 = (Po + PiPo + P2P1P0 + P3P2P1P0 H ) (1 - 5) , 

which follows 

Po + P1P0 + P2P1P0 + P3P2P1P0 H 

<7 = -. 

1 + (Po + P1P0 + P2P1P0 + P3P2P1P0 H ) 

Thus this gives that for k > 1 

= Pk-lPk-2 ■ ■ ■ PlPO + PkPk-lPk-2 ■ ■ ■ PlPO + Pk+lPkPk-lPk~2 ■ ■ ■ PiPo j 

1 + (Po + PiPo + P2P1P0 + P3P2P1P0 H ) 

5.3 The M/M/l queue with server multiple vacations 

We consider an M/M/l queue with server multiple vacations, where the arrival, service and 
vacation rates are A, \i = 1 and 9. The vacation process is based on the multiple vacation 
policy: When there is not any customer at one server and its buffer, it immediately takes 
a vacation and keeps taking vacations until it finds at least one customer waiting in the 



= Pk \^k - TTfe+l) 

= g e (0, 1) and p k ^i = Xk-i/fJik for k > 1. 
-TTi) =p (l-g) 
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server or its buffer at the vacation completion instant. The arrival, service and vacation 
processes are independent of each other. 

Let N (t) be the number of customers in the queueing system at time t, and 



at) 



V, if the server is taking a vacation at time t, 
W, if the server is working at time t. 

Then {(£ (t) , N (i)) : t > 0} is a Markov chain on a state space E = {{V, k) , (W, I) : 
k > 0,1 > 1}. We write 



and 



Qv,i(t) = P{s(t) = v,N(t)>i}, i>o, 



Qw,k(t)=P{C(t) = W,N(t)>k}, k>l; 



nv,k = , hm Q Vk (t), k > 0, 

= lim Qwj (t), 1>1. 
t— >+oo 



Then we obtain 



(nW,l — KW,2) - 0lT V) i = 0, 

A (nv,k-i ~ Kv,k) ~ d^v,k = 0, k > 1, 
Note that 7rv,o = 1 — A and 71^1 = A, thus using (|45l) we obtain 



(44) 
(45) 
(46) 



A 



A + 



(1-A), £;>0, 



and from (I44D that 



7TTV,2 = A 



A0 
A + ( 



(1-A). 



Using ttw,i = A and = A — \9 (1 — A) / (A + 6), it follows from (|4"o) that for k > 3 



^W.fc = 7tW,fc-l — A (7TWfc-2 — TTW.fc-l) — ^"V,A-1) 



(47) 



which can be computed iteratively. 
Let 

/-(1 + A) A 

1 -(1 + A) 



Q 



l 



:i+a) a 
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Then using ([37]) we obtain 

(TTw,2, K\V,3, 7TW,4) • • •) Q = - (^W,l + 0KV,2> ^V,3, G^VA, . . .) 

Let 

R = \, G = l. 



(48) 



Then 



and 



Thus we obtain 

(TTw,2,^W,3,^W,4y ■ 

Note that 



and 



U k = -(l + \) + R = -l, k>0, 
U D = diag (-1, -1, -1, -1, . . .) , 

/0 A \ 
A 
A 



Ru 



\ 



■ J 



( 



G L 





1 
1 



\ 



V 



/ 



(Xttw,i + 0ttv,2, 9nv,3, O^va, ■ ■ •) Q max 

(Xn W ,l + 07T V ,2, 07iy, 3 , 9TTVA, . . .) (I - G^ 1 (/ - fltf) 



-1 



(I-Gl) 



-i 



1 

1 1 
1 1 1 
1111 



\ 



V 



(I-Rur 1 



I 1 A A 2 A 3 
1 A A 2 
1 A 
1 
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for k > 2 we can obtain 

k~ 2 oo 



= A fc + (1 - A) 



'E 


A* 




i=0 




l=k—i 


A 2 


1 - 


-( " 








fc-1 



5.4 The M/M/l queue with repairable server 

We consider an M/M/l queue with repairable server, where the arrival and service rates 
are A and //, respectively. The life time of the server is exponential with failure rate a > 0. 
Once the server failed, it immediately is repaired, and the repair time is exponential with 
repair rate (3. The repaired server is the same as the new one. We assume that all the 
random variables defined above are independent of each other. 

For this M/M/l repairable queue, we denote by N (t) and C (t) the number of cus- 
tomers in this queueing system and the state of the server at time t > 0, respectively, 
where N (t) = 0, 1, 2, . . ., and C (t) = W for server working or R for server repair. It is 
easy to see that {(N (t) , C (t)) : t > 0} is a Markov chain. For k > and / > 1, we write 



and 



and 



Then we obtain 



Qw,k (t) = P{C(t) = W,N(t)>k} 



Qr,i (t) = P{C(t) = R,N(t)>l}; 
Kw,k = , lim Q Wk (t) 



kr,i = lim Q R i (t) . 



KWfl + TTR,1 = 1 (49) 

- air w ,i + Pttr,i = 0, (50) 

for k > 1 

A {irw,k-i ~ Kw,k) ~ M {^w,k ~ ^w,k+i) ~ aTr Wjk + I3tt r ^ = 0, (51) 

for I > 2 

A {ttr,i-i - 7Tr,i) + an w> i - (3n Rt i = 0. (52) 
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It follows from ([51]) that 

oo oo 



k=l k=l 

and from ([5"2"]) that 

OO OO 

fc=2 fc=2 

which, together with ([50]) . leads to 

oo oo 

A7T fij i + a 7Tiy ifc - 7TR ifc = 0. (54) 

fc=l k=l 

Using ([53]) and ([54"]) . we obtain 

Avtvk,o - ^ttvk.i + = 0. (55) 

It follows from (39]), ([50]) and ([55]) that 

A a 

A 

7TW.1 = — = P 

A* 

and 

A a 
MP 

It follows from (ED) and O) that for k > 2 



A + ^/ + q A f3 

KW,k = 7TW,fe-l 7r W,fc-2 7Til,fc-l 

and 

q A 
KR,k = x , a ^W,k + , , a ^R,k-l- 

A + p X + p 

Therefore, nw,k and itr^ for /c > 2 can be computed iteratively. 

Remark 1 Since the M/M/l queue with repairable server may be described as a level- 
independent QBD process, the tailed probabilities 7rw,k for k > and ttri for I > 1 can be 
given by the matrix- geometric solution (EJ) or [W\). 
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6 Concluding remarks 

This paper discusses the tailed probabilities of queueing processes, such as, the QBD pro- 
cesses and Markov chains of GI/M/1 type and of M/G/l type, and provides some effective 
and efficient algorithms for computing the tailed probabilities by means of the matrix- 
geometric solution, the matrix-iterative solution, the matrix-product solution and the two 
types of iiG-factorizations. Also, we consider four queueing examples: The M/M/l retrial 
queue, the M(n)/M(n)/1 queue, the M/M/l queue with server multiple vacations and the 
M/M/l queue with repairable server, where the M/M/l retrial queue is given a detailed 
discussion, while the other three queues are analyzed simply. It is seen from the four 
queueing examples that the method of this paper can be applied to deal with more gen- 
eral queues including the MAP/PH/1 queue, the GI/PH/1 queue and the BMAP/SM/1 
queue. 

The results given in this paper are very useful in the study of large scale stochastic net- 
works with resource management, such as, supermarket models and work stealing models. 
Also, it will open a new avenue to helpfully analyze the tailed probabilities of many large 
scale stochastic networks when applying differential equations and mean-field limits. 
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